129 research outputs found
SHADHO: Massively Scalable Hardware-Aware Distributed Hyperparameter Optimization
Computer vision is experiencing an AI renaissance, in which machine learning
models are expediting important breakthroughs in academic research and
commercial applications. Effectively training these models, however, is not
trivial due in part to hyperparameters: user-configured values that control a
model's ability to learn from data. Existing hyperparameter optimization
methods are highly parallel but make no effort to balance the search across
heterogeneous hardware or to prioritize searching high-impact spaces. In this
paper, we introduce a framework for massively Scalable Hardware-Aware
Distributed Hyperparameter Optimization (SHADHO). Our framework calculates the
relative complexity of each search space and monitors performance on the
learning task over all trials. These metrics are then used as heuristics to
assign hyperparameters to distributed workers based on their hardware. We first
demonstrate that our framework achieves double the throughput of a standard
distributed hyperparameter optimization framework by optimizing SVM for MNIST
using 150 distributed workers. We then conduct model search with SHADHO over
the course of one week using 74 GPUs across two compute clusters to optimize
U-Net for a cell segmentation task, discovering 515 models that achieve a lower
validation loss than standard U-Net.Comment: 10 pages, 6 figure
C-CLIP: Contrastive Image-Text Encoders to Close the Descriptive-Commentative Gap
The interplay between the image and comment on a social media post is one of
high importance for understanding its overall message. Recent strides in
multimodal embedding models, namely CLIP, have provided an avenue forward in
relating image and text. However the current training regime for CLIP models is
insufficient for matching content found on social media, regardless of site or
language. Current CLIP training data is based on what we call ``descriptive''
text: text in which an image is merely described. This is something rarely seen
on social media, where the vast majority of text content is ``commentative'' in
nature. The captions provide commentary and broader context related to the
image, rather than describing what is in it. Current CLIP models perform poorly
on retrieval tasks where image-caption pairs display a commentative
relationship. Closing this gap would be beneficial for several important
application areas related to social media. For instance, it would allow groups
focused on Open-Source Intelligence Operations (OSINT) to further aid efforts
during disaster events, such as the ongoing Russian invasion of Ukraine, by
easily exposing data to non-technical users for discovery and analysis. In
order to close this gap we demonstrate that training contrastive image-text
encoders on explicitly commentative pairs results in large improvements in
retrieval results, with the results extending across a variety of non-English
languages.Comment: 11 Pages, 5 Figure
Unfooling Perturbation-Based Post Hoc Explainers
Monumental advancements in artificial intelligence (AI) have lured the
interest of doctors, lenders, judges, and other professionals. While these
high-stakes decision-makers are optimistic about the technology, those familiar
with AI systems are wary about the lack of transparency of its decision-making
processes. Perturbation-based post hoc explainers offer a model agnostic means
of interpreting these systems while only requiring query-level access. However,
recent work demonstrates that these explainers can be fooled adversarially.
This discovery has adverse implications for auditors, regulators, and other
sentinels. With this in mind, several natural questions arise - how can we
audit these black box systems? And how can we ascertain that the auditee is
complying with the audit in good faith? In this work, we rigorously formalize
this problem and devise a defense against adversarial attacks on
perturbation-based explainers. We propose algorithms for the detection
(CAD-Detect) and defense (CAD-Defend) of these attacks, which are aided by our
novel conditional anomaly detection approach, KNN-CAD. We demonstrate that our
approach successfully detects whether a black box system adversarially conceals
its decision-making process and mitigates the adversarial attack on real-world
data for the prevalent explainers, LIME and SHAP.Comment: Accepted to AAAI-23. 9 pages (not including references and
supplemental
Network intrusion detection with semantics-aware capability
© 2006 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Pre-print of article that appeared at the 2 nd International Workshop on Security i
- …